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Abstract 

To optimize rotated, multidimensional constellations over a single-input, single-output Rayleigh 
fading channel, a family of rotation matrices is constructed for all dimensions which are a power of 2. 

This family is a one-parameter subgroup of the group of rotation matrices, and is located using a gradient 
descent scheme on this Lie group. The parameter defining the family is chosen to optimize the cutoff 
rate of the constellation. The optimal rotation parameter is computed explicitly for low signal-to-noise 
ratios. 

These rotations outperform full-diversity algebraic rotations in terms of cutoff rate at low SNR 
(signal-to-noise ratio) and bit error rate at high SNR in dimension n = A. However, a QAM (quadrature 
amplitude modulation) constellation rotated by such a matrix lacks full diversity, in contrast with the 
conventional wisdom that good signal sets exhibit full diversity. A new notion of diversity, referred to 
as local diversity, is introduced to attempt to account for this behavior. Roughly, a locally fully diverse 
constellation is fully diverse only in small neighborhoods. A local variant of the minimum product 
distance is also introduced and is shown experimentally to be a superior predictor of constellation 
performance than the minimum product distance in dimension n = A. 
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I. Introduction 

A. Rotated Constellations and the Cutoff Rate 

Communication over a wireless ehannel typically involves transmitting a signal which is subse¬ 
quently affected by fading and noise. Ensuring that such transmission is done reliably requires an 
energy-effieient eonstellation resistant to these effects. It is known that rotating constellations is 
an effeetive way to eombat the effect of fading, while maintaining the energy of eaeh eodeword. 
Rotations effeetively combat fading by increasing the diversity of the eonstellation. 

Numerous algebraie and number-theoretie teehniques exist to eonstruct fully-diverse rotated 
eubie lattices with good minimum product distance |[I1, [|3, [l3]|, of whieh a finite subset is 
seleeted to serve as the eonstellation. These algebraie methods are effeetive at produeing fully- 
diverse eonstellations whieh have good performance in terms of error rate. However, the rigidity 
of the algebraie eonstruetions imposes a laek of flexibility in adjusting the partieular rotation 
matrix in the presence of, for example, an outer error-eorreeting code or varying SNR (signal- 
to-noise ratio). Furthermore, the number-theoretie eonstruetions of [|3l assume high SNR and 
that the signal set is essentially infinite, that is, eonsists of an entire lattiee in M”. 

We investigate potential improvements in the algebraic method of (Si by dropping these 
assumptions and investigating rotations via a finer and SNR-dependent objeetive function con¬ 
trolling constellation performance. This objeetive funetion is an approximation of the eapacity 
of the eonstellation known as the cutoff rate, which has a long history of being used as design 
criterion, but almost exelusively for erasure and AWGN channels; see the classieal references 
DU, DSl and the referenees therein. It has been utilized much less for the fading ehannel model 
we eonsider, in favor of eoarser, more algebraie measures of constellation performance. 

Rotating eonstellations to improve CM (eoded modulation) eapacity has been investigated 
previously in DUl, D3, DSl. Our work is partially inspired by DTI, in whieh the authors approximate 
the CM capaeity by the eutoff rate, and use the latter to construet good rotation matrices for 
4-QAM and 16-QAM (quadrature amplitude modulation) eonstellations in and C®. The eutoff 
rate was also eonsidered in eonjunetion with eonstellation design in DHl, DU. Our goal is to apply 
the general method of DU of considering rotations whieh attempt to maximize the cutoff rate to 
the ehannel model of DU, for whieh one often eonstructs algebraie rotations. The eutoff rate for 
the channel we eonsider is a simple rational funetion of the coordinates of the eonstellation points 
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and therefore, unlike working with the CM eapaeity direetly, does not require one to numerieally 
eompute numerieally unstable unbounded integrals during optimization. In partieular, the eutoff 
rate is simple enough to be suseeptible to the methods of IfTOll for numerieal optimization over 
rotation matriees, whieh we employ to investigate optimal rotation matriees. 

B. Non-Uniform Constellations 

By a non-uniform eonstellation, one typieally means a “perturbation” of a standard QAM 
eonstellation, whieh heuristieally behaves like a eloser approximation of a Gaussian distribution 
than the standard “square” QAM eonstellations. Non-uniform eonstellations for the AWGN 
ehannel have been studied in [fTTll . wherein the authors perform SNR-dependent optimization of 
sueh eonstellations, and in ifT^ with the goal of optimizing the BICM (bit interleaved eoded 
modulation) eapaeity. Non-uniform eonstellations have been shown to have good error rate per- 
formanee for the Gaussian lfT3l and Rayleigh lfT4l ehannels. Furthermore, non-uniform 64-QAM 
and 256-QAM eonstellations have been eonsidered in DVB-NGH (Digital Video Broadeasting 
- Next Generation Handheld) implementation lUSll due to the improvement in eapaeity, and are 
under eonsideration for inelusion in the ATSC (Advaneed Television Systems Committee) 3.0 
industry standard ifT^ . 

On the other hand, the DVB eonsortium has established rotated QAM eonstellations as a 
part of the DVB-T2 industry standard [fTTll . and rotated QAM eonstellations in and as 
part of the DVB-NGH (next-generation handheld) standard ifTSli . A eoneise summary of sueh 
modulation teehniques ean be found in IfT^ §11]. A natural question is whether eombining non¬ 
uniformity and rotations ean offer an additional inerease in eapaeity, and thus one would like to 
know how to find good rotations for arbitrary eonstellations, not just traditional M-QAM. As 
optimal non-uniform eonstellations ean ehange with the SNR, and our goal is partly to rotate 
sueh eonstellations, we seek a rotation method whieh is SNR-dependent as well. Working with 
the CM eapaeity and eutoff rate also demands that one eonsiders SNR-dependent rotations, 
sinee both of these objeetive funetions depend on the SNR. As we shall see in Seetion fllll 
optimal rotations ean be highly SNR-dependent and optimizing rotations with respeet to SNR 
ean improve error rate performanee. 
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C. Summary of Main Contributions 

We present a general method for eonstrueting good rotations of arbitrary eonstellations for 
single-input, single-output Rayleigh fast fading ehannels in dimensions whieh are a power of 
2. Construeting rotations in arbitrarily high dimensions means abandoning explieit parameter- 
izations of sueh matriees, sinee sueh parameterizations beeome non-eanonieal and unwieldy 
as the dimension of the ambient spaee inereases. The eolleetion of all rotations of is the 
special orthogonal group SO{n) whieh has dimension {nf — n)/2 as a real manifold, meaning 
any parameterization of n x n rotation matriees requires at least — n)/2 variables. We will 
present the neeessary mathematieal framework in Seetion 0 

Our approaeh to eonstrueting optimal rotation matriees differs largely from previous work on 
the subjeet, in that we effeetively replaee explieit parameterizations of rotation matriees with the 
matrix exponential map 

exp : so(n) —)■ SO{n) (1) 

where so(n) is the Lie algebra of all skew-symmetrie matriees. The general mathematieal 
framework of Lie groups and Lie algebras allows us to eonstruet well-performing families Q 2 fe (f) 
of 2^-dimensional rotation matriees for all k, whieh we view as one-parameter subgroups of 
SO (2^). The problem of optimizing over all (n^ — n)/2 parameters defining a rotation matrix 
is thus redueed to optimizing over just a single parameter t G [0,7r/2], whieh is easily done by 
exhaustive seareh and ean be entered to the given SNR and eonstellation. 

The flexibility of the parameter t is also benefieial if one wishes to further optimize due to 
the presenee of an outer error-eorreeting eode, a partieular deeoding algorithm, a partieular bit 
interleaver, ete. In faet this is essentially what is done in the DVB-NGH standard ifTSl §6], whieh 
eonsiders the (transpose of the) same family of matriees Q 4 (f) whieh we eonstruet explieitly, 
and optimizes the parameter t to suit the outer LDPC eode rate and modulation order. By 
generalizing this eonstruetion to any dimension whieh is a power of 2, we hope to provide 
robust and adaptive families of rotation matriees to suit a number of applieations. In general, 
we view our method as an alternative to the algebraie and number-theoretie eonstruetions whieh 
aeeounts for the finiteness of the SNR, the boundedness of the eonstellation, and is adaptable 
to a number of potential objeetive funetions whieh eontrol system performanee. We hope to 
generalize our method to the MIMO setting, and analogously improve upon the performanee of 
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division algebra codes for MIMO Rayleigh fading channels ||T91 . lattice codes for the Rician 
fading channel ll20ll . and full-diversity unitary precoding techniques for MIMO channels [f2T]| . 
Our adaptive approach may also be useful in newer protocols such as SCMA (sparse code 
multiple access) ll^ . in which algebraic rotations are proposed, but the complexity of SCMA 
codebook design may demand more intricate constructions in the future. 

This article is intended to summarize, expand upon, and provide a more satisfying theoretical 
framework for the results of the conference papers lf2^ . Il24ll by the current authors. In particular, 
the family of rotation matrices Q 2 fc(^) we consider was discussed in If24]| but we present here 
a simple, explicit formula in Section IVI-BI for their construction. We find in Section IVI-CI the 
optimal rotation within the family Q 2 fc(f) in the low SNR regime and confirm our results via 
experiment. We investigate in Section IIV-AI the diversity order of our rotated constellations and 
show in Section IVII-AI that despite the lack of full diversity, they outperform the fully-diverse 
algebraically-rotated constellations of [|3l| in terms of both cutoff rate (at low SNR) and error 
performance (at high SNR) in dimension 2^ = 4. We attribute this to a property we call local 
full diversity, which describes constellations which may not be fully diverse, but are so if one 
restricts to small neighborhoods. Our experiments indicate that constellations which enjoy this 
less stringent property can offer modest gains over fully diverse signal sets. This is in contrast 
with the conventional wisdom presented in Q that optimal signal sets have full diversity. 

II. Coded Modulation Capacity and Cutoff Rate 

We consider the general problem of constructing good constellations for Rayleigh fast fading 
single-input single-output (SISO) channels for transmission over n time instances. We assume 
perfect channel state information at the receiver, and as in [|3l §2], by performing phase cance¬ 
lation at the receiver, separating real and imaginary parts, and employing a bit interleaver, we 
can model the channel as 

y = hx -P z (2) 

where 

• X = (xi,..., XnY G M" is a column vector of transmitted data, selected uniformly at random 
from a finite constellation Tf C M" of size \X\ = 2'^, 

• h = diag(/ii,..., hn) is a real diagonal nxn matrix with hi a Rayleigh random variable (i.e. 
the norm of a circularly symmetric complex Gaussian random variable) with E(/i^) = 1, 
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• z = (zi,..., ZnY G M” is a noise vector with Zi a real zero-mean Gaussian random variable 
with variance Nq, and 

• y G M" is the received vector. 

We assume that the constellation is subject to an energy constraint of the form 

E||x|p = |^y^||x||^ = i’ (3) 

' ' xex 

for some fixed P, where \X\ denotes the cardinality of X. Let us define the average energy per 
bit Eh by P/ log 2 (|A’|). We will present simulation results in terms of E^/Nq in decibels, but we 
will slightly abuse terminology and occasionally refer to this quantity as the SNR (signal-to-noise 
ratio) as well. 

If T is any n x n real matrix (in all cases we consider T will either be a channel matrix or 
a rotation matrix), then by the notation TX we mean the constellation 

TX = {Tx I X G A’} C (4) 


A. Definitions and Basic Notions 


In this section we recall some familiar formulas for the CM capacity of an n-dimensional 
constellation A" C of size \X\ = 2^. This discussion follows that of |I71, but for the slightly 
different channel model dll). If we set h = in dH), the mutual information of the output y 
and the constellation X is 


i(y,x) = \og,A\x\)-Y^ [ 

xex -'I* 


exp(-||y-x||V7Vo) 


lAKvriVo) 


X logs 


exp 


x'eA" 

xVx 


x|p — ||y — x'lp 

iVn 


(5) 


dy 


If h = diag(/ii,..., hn) is a fixed fading matrix, then the conditional mutual information of y 
and X given h is 


J(3;,A;h) = J(y,hA). 


( 6 ) 


One defines the coded modulation capacity (CM capacity) of X by taking the expectation over 
all h: 


G^“(A) = Eh/(y,A;h) 


( 7 ) 
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For fixed n and Eh/No, one would ultimately like to optimize C™(A’) over all eonstellations 
X, but the large amount of numerieal integration required to eompute makes this 

intraetable, espeeially with inereasing dimension and eonstellation size. In faet, even evaluating 
^CM(^) numerieal preeision required to apply our optimization teehniques seems out 

of reaeh, even for small eonstellations in small dimensions. Our first step is therefore to replaee 
with a more traetable objeetive funetion. 


B. The Cutoff Rate 

It is known [|71, (El, [l25ll that one ean bound I{y,X-,h.) below by a quantity i?o(A’; h), whieh 
for the model dH) is given by 

-||h(x-y)|P^l 

8iv^ )\ 

where reeall that \X\ = 2*^, so that eaeh eonstellation point eneodes q bits. One ean now define 


Ro{X-,h) = q- logs 


1 + 2"^ exp 


Ro{X) = Ey,Ro{X;h) 


(9) 


and it follows trivially that C^^{X) > Ro{X). 

To establish numerieal teehniques whieh are independent of eomputing an expeetation over all 
h, one uses Jensen’s inequality to further lower bound Ro{X) by the cutoff rate R{X), defined 

by 


R(X) = q- logs 


i+2-» y] n 


x^yeA” i=l 


1 


1 I 

87Vo J 


( 10 ) 


For some fixed x ^ y E X, the eorresponding summand in (fTOl) is related to the pairwise error 
probability of eonfusing y for x at the reeeiver O. Our goal is now to optimize our eonstellation 
X by treating R{X) as an objeetive funetion we wish to optimize for a given Eh/No, dimension 
n, and modulation order \X\. We briefly remark that most authors 0, [|^ use the term “eutoff 
rate” to refer to the quantity Ro{X), but we reserve the term for R{X) to simplify terminology, 
sinee it is the main objeetive funetion we eonsider. 

Studying the eutoff rate eliminates the need to perform the numerieal integration neeessary to 
evaluate C^^{X), whieh ean be eomputationally demanding or even infeasible, espeeially sinee 
the integral is unbounded and the integrand is numerieally unstable. Furthermore, the use of 
Jensen’s Inequality has eliminated eomputing Eh(-)> whieh would further require eomputing a 









second numerically unstable integral, or measuring Ro{X) via Monte Carlo simulations. Even 
small improvements in Rq{X), and R{X) can introduce non-trivial benefits in terms 

of, for example, error performance, and small gains in the former two quantities are necessarily 
very difficult to measure, especially for larger dimensions and modulation orders. Hence we will 
almost exclusively measure performance in terms of R{X). Note that the dependence of R{X) 
on the noise variance Nq allows us to tailor our constellations to particular SNR regimes. 

III. Non-Uniform QAM Constellations in 

Let us begin by studying the model Q when n = 2. The purpose of this section is to in¬ 
troduce non-uniform constellations and demonstrate their utility. Non-uniform QAM (NUQAM) 
constellations have been included in the DVB-NGH standard [fTSl §6] for modulation orders of 
M = 64, 256, wherein their defining parameters have been optimized according to the rate of 
an outer LDPC code. Here we demonstrate how optimal rotations of constellations vary when 
one changes both the SNR and the non-uniformity parameters, which helps motivate our more 
intricate constructions in later sections when we increase the value of n. 

A. Optimal Non-Uniform Constellations 

Let g > 4 and consider a set of parameters a = (ai,..., 029 / 2 - 1 ) £ ^ with Oj > 0 for 

all i. By an M-NUQAM constellation X{a) C we mean the direct product of the set 

{—029/2-1, . . . , —Oil, Oi, . . . , O29/2-1} (11) 

with itself, such that |A’(o)| = M. Lor example, when g = 4 and o = (1,4), we have the 
16-NUQAM constellation 

A'(l, 4) = {(±1, ±1), (±1, ±4), (±4, ±1), (±4, ±4)}. (12) 

The Oj should be interpreted as perturbed versions of the normal parameters defining the 
standard M-QAM constellations. We will refer to the Oj as the non-uniformity parameters. 
Such constellations are known to increase capacity at lower SNR ranges ifTSll . 

Lor a fixed constellation size and Eb/No, we optimize the non-uniformity parameters o by 
performing steepest descent on the objective function R{X{a)). In Table H] we give examples of 
some such numerically-optimized NUQAM constellations, and in Lig. [T]we plot an example of 
a 1024-NUQAM constellation. 


TABLE I 


Some Example M-NLFQAM Constellations 


Ei,/No (dB) 

M 

optimal a = (ai,... , 029 / 2 - 1 ) for M-NUQAM 

8 

16 

(0.9732, 3.0088) 

12 

64 

(0.9179, 2.7927, 4.8112, 7.2257) 

15 

256 

(0.8912, 2.6844, 4.5119, 6.4022, 8.3956, 10.5573, 13.0147, 16.1037) 


30 

20 


o 10 

w 
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Fig. 1. Scatter plot of a 1024-NUQAM constellation in R^, optimized for Eh/No = 20 dB. 


Of course, one eould maximize the eutoff rate of X by performing steepest deseent on every 
coordinate of every signal point. However, when \X\ = 2^^, for example, this approach would 
require optimization over 2-2^^ = 8192 real parameters, whereas optimizing with respect to 
non-uniformity parameters requires only = 32 real parameters. In the next subseetion 

we shall see how to further improve the cutoff rate by the addition of a single real parameter to 
the search space, namely one governing rotation. 

B. Optimal Rotation Angles 

Let us study how to further improve the cutoff rate of constellations in by rotating them. 
While this offers large benefits in terms of rate, we note that rotated eonstellations are neeessarily 
more diffieult to decode, since one cannot decode the coordinates independently. We optimize 
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the rotation angle of a constellation X by considering the objective function 

cos(f) sin(f) 

— sin(f) cos(f) 

and performing a brute force search over the interval f G [0,7r/2]. 


R{Q2{t)X), Q2(f) = 


(13) 



10 15 20 25 30 35 

E,/N„ (dB) 


^<-64-QAM 

^<-256-QAM 

^<-1024-QAM 

-^64-NUQAM 

-&256-NUQAM 

- v-1024-NUQAI\/l 



Fig. 2. On the left, we plot optimal rotation angles for M-QAM and M-NUQAM constellations in for M = 64, 256, and 
1024. The horizontal black line represents the angle corresponding to the optimal two-dimensional algebraic rotation matrix A 2 
1261 . On the right, we plot the bit error rate of the 64- and 256-NUQAM constellations, rotated by the optimal rotation matrix 
Q 2 (fopt) as well as the algebraic rotation A 2 . 


To motivate the need for constellation- and SNR-dependent rotations, we plot in Fig. [2] the 
rotation angle fopt £ [0,7r/2] which optimizes the objective function in (fT3l) as a function of 
Eb/No. The NUQAM constellations used were optimized for each value of Eb/No and then 
optimal rotation angles were considered. It is clear from Fig. [2] that the optimal rotation angle 
depends on the modulation order, the SNR, and the uniformity of the constellation. 

Also in Fig. [2] we plot the bit error rate for the 64- and 256-NUQAM constellations, rotated by 
the optimal matrix Q 2 (fopt) as well as the optimal algebraic rotation A 2 [|2^ . Bit strings of length 
at least 10® were modulated to the constellation points at every value of Eb/No, according to a 
Gray labeling. We see that the gain obtained by rotating on a per-SNR basis can be as high as 1.5 
dB at high values of Eb/No for both NUQAM constellations. While the operating range for the 
SNR is quite high, we are ultimately interested in comparison with the algebraic lattices of llT9l 
which are designed for the very high SNR regime. Thus the main point is not the magnitude of 
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the gains but rather that there are any gains at all, especially in the very high regime for which 
algebraic rotations are often considered optimal. 

These plots provide motivation for finding SNR- and constellation-dependent rotation matrices 
for constellations in as well as in larger ambient spaces. The algebraic and number-theoretic 
methods of a, m of constructing good rotated Z’^-lattices necessarily fall somewhat short 
of this goal since NUQAM constellations are not subsets of lattices, and furthermore algebraic 
lattices are not SNR-dependent. 



20 25 

E,/N„ (dB) 



Fig. 3. On the left, we plot the improvement in R{X) for non-uniform (NUQAM), rotated (RQAM), and rotated non-uniform 
(RNUQAM) constellations over their unrotated, uniform counterparts. On the right, we plot the experimentally measured increase 
in Ro{X) for 64-RQAM and 256-RQAM constellations. 


In Fig. [3] we plot the overall increases 

AR{X) = R{X) - /^(T'qam), Ai?o(^) = R^{X) - Ro{Xqam) (14) 

in cutoff rate for various constellations X, over their uniform, unrotated counterparts of the 
same modulation order, denoted by T’qam- In measuring the increase in R{X), we used non- 
uniform (NUQAM), rotated (RQAM), and rotated non-uniform (RNUQAM) constellations, for 
M = 64, 256, and 1024, over their uniform, unrotated counterparts. For the NUQAM and 
RQAM constellations, the non-uniformity and rotation parameters were numerically optimized by 
performing steepest descent on the cutoff rate. The RNUQAM constellations were first optimized 
with respect to the non-uniformity parameters, and then with respect to rotation. Notice that non¬ 
uniformity and rotation can improve the cutoff rate of the constellation at different values of 
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Ei^/Nq. For example at Eh/No = 20 dB, 1024-QAM benefits mueh more from introdueing non¬ 
uniformity than from rotation. Lastly we note that the benefit of rotation is most pronouneed in 
the finite SNR regime. 

The quantity ARq{X) was eomputed for 64-RQAM and 256-RQAM by experimentally av¬ 
eraging the value of Ro{X] h) — i?o('^QAM; h) over 2 * 10^ ehannels h. Here optimal rotations 
were eomputed on a per-SNR basis, so that X was ehosen to be the rotated version of T^qaivi 
whieh maximizes R{X). As one ean see by eomparing the plots, maximizing R{X) results in a 
signifieant inerease in Ro{X) as well, and thus we inelude these simulation results to empirieally 
demonstrate the strong correlation between these two objective functions. However, due to the 
difficulty of computing Ro{X) even for low dimensions, we will forego measuring constellation 
performance in terms of Ro{X) and only present further simulation results in terms of R{X). 

IV. Local Diversity 

Note that in Fig. [2] the optimal angle for a number of constellations and SNR values was 
t = 45°, which does not offer full diversity (in the sense of (Si and defined below) in the case 
of a rotated QAM constellation. Thus at finite SNR, constellations which are not fully diverse 
have the potential to outperform fully diverse constellations. In this section we define a subtler 
notion of diversity which seeks to explain this phenomenon. 


A. Full Diversity and Minimum Product Distance 


Let X C M” be any constellation. If we let Nq —)■ 0, that is, consider the high SNR regime, 
then we could further approximate the cutoff rate (fTOl) by 


R{X) ^ q - log 2 [1 + 2-'^^o] 


(15) 


where 


So= E n 


8Nn 


x^yGV i=l 


[Xi 


Vi) 


(16) 


To maximize R{X), then, one wishes to minimize Sq. As a first attempt at doing so, one 
maximizes the number of terms in each of the above products. For that we consider the diversity 
order L, defined by 


L{X)= min 

XT^yeV 


Xi 7^ Vi} 


(17) 
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In other words, L{X) measures the minimum number of coordinates in which two signal points 
differ. When L{X) = n one says that X has/w/Z diversity, which is often a desirable property 
and can be effectively achieved via algebraic methods [I3]|. 

As a sharper measure of constellation performance, one can then consider the minimum product 
distance dp{X) defined by 

n 

dp{X) = min IT \xi - yi\ (18) 

2=1 

At high SNR design criteria typically reduces to choosing an X which maximizes dp{X), for 
which the number-theoretic constructions of Q are effective. 

B. Local Cutoff Rate and Local Diversity 

The notion of full diversity can be somewhat restrictive. Indeed, it may not matter that much 
that all of the coordinates of x and y are distinct if they are very far from one another in 
Euclidean distance. Thus rate and error performance should depend only on local neighborhoods 
of the constellation and not on its global structure. We loosen the notion of full diversity in the 
following way to account for this fact. 

Let X G TZ and fix a radius 0 < r < cxo. Define the ball of radius r at x by 

B(x,r) = {y e A" I ||x-y ||2 < r}. (19) 

The intuition behind the following definition is that the summands which dominate the summation 
appearing in (fTOl) are those for which the points x 7 ^ y are close in Euclidean distance. 
Definition 1: Given a radius 0 < r < 00 , we define the local cutoff rate R{X,r) of X by 

R{X,r) = q-log fil + 2-‘>S{r)) (20) 

where 

sw = E E n . , (!.-,.)■ ■ (21) 

xeA” yes(x,r) i=l ^ 8 No 

The accuracy of R{X, r) to predict the value of R{X) depends on the value of r, and the r one 
wishes to consider may in turn depend on the SNR. Eor r = cxo we have R{X, 00 ) = R{X). One 
of the main advantages of considering local cutoff rates is that one can often find explicit provably 
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optimal rotations, as we shall see. These rotations will not result in fully diverse eonstellations, 
but they will be “loeally” fully diverse in the following sense. 

Definition 2: Given a radius 0 < r < cxo, the local diversity order of X is 


L{X,r) = min min \ Xi 7 ^ yi} 

x£X yGB(x,r') 


( 22 ) 


If L{X, r) = n for some radius r then we shall say that X has local full diversity. If L{X, cx)) = n 
we shall say that X has global full diversity. 

The property of loeal full diversity of eourse depends on the partieular radius, whieh we shall 
always make explicit. Global full diversity is equivalent to full diversity in the traditional sense 
of (fTTI) . but we append the adjective ‘global’ to emphasize the distinction. 

For example, consider the constellation 



(23) 


One sees easily that L{X, 2) = 2 but L{X, 00 ) = 1. Hence X is locally fully diverse but not 
globally fully diverse. In fact, X is the standard 4-QAM constellation in rotated by f = 45°, 
which the experiments of the previous section show is often an optimal rotation for constellations 


in 


Loosely speaking, if the SNR is such that the large majority of errors cause constellation points 
to be confused only for others within a ball of radius r, then the performance of a locally fully 
diverse constellation should be indistinguishable from that of a globally diverse constellation. 
The above definitions reflect the fact that error performance is largely a local invariant of a 
constellation. 

We can compare two locally diverse constellations at high SNR using the following variant 
of the minimum product distance. 

Definition 3: Given a radius 0 < r < 00 , the local minimum product distance of X is 


n 


dp{X, r) = min min Xi — pi 


= mm mm 

'x.&X y£B{x,r) 


xGX y(iB{-x.,r) 



(24) 


For r = cx we have the obvious equality dp{X,oo) = dp{X). In Section IVII-AI we will show 
that the local minimum product distance can sometimes predict the relative performance of two 
constellations when dp{X) cannot. 
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V. The Special Orthogonal Group SO in) 

The large gains in eutoff rate offered by rotations at finite SNR for two-dimensional eon- 
stellations motivate us to find good rotations for higher-dimensional eonstellations, whieh are 
tailored for a given SNR and eonstellation. To provide the proper mathematical framework, we 
first discuss the natural habitat of higher-dimensional rotations. 

A. Basic Definitions 

To properly describe the families of rotation matrices we construct, we need to discuss the 
structure of the special orthogonal group SO{n) of all n-dimensional rotation matrices. The 
group SO{n) of rotations of M” is defined by 

SO{n) = {Q G GL{n) \ QQ' = I„, det(Q) = 1} (25) 

where GL{n) is the group of all invertible real n x n matrices. The dimension of SO{n) as a 
manifold is {n? — n)/2, which can be thought of as the minimum number of parameters required 
to describe an n x n rotation matrix. The special orthogonal group is a Lie group, which is 
both a group and a manifold such that the group operations are continuous with respect to the 
manifold structure. As a general reference for the theory of Lie groups we recommend GTll . 

The Lie algebra so{n) of SO{n) is the tangent space at the identity matrix G SO{n), and 
thus is a real vector space of dimension (n^ — n)/2. We have the following convenient explicit 
description: 

so{n) = {A G M{n) \ - A = A*} (26) 

where M{n) is the set of all n x n real matrices. We pass from the Lie algebra to the Lie group 
using the exponential map exp : so{n) SO{n), defined by the familiar power series 

exp (A) = -f- A -f -!-■■■ (27) 

One can verify easily that the exponential series converges for all A. There also exists a logarithm 
map log : SO{n) 5o{n), defined by a similar power series and satisfying exp(log(A)) = A. 
It is a basic fact that the exponential of a skew-symmetric matrix is a rotation matrix, and that 
the logarithm of a rotation matrix is skew-symmetric. 
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A one-parameter subgroup of SO{n) is the image of a continuous group homomorphism 
M —)■ SO{n). One can show all one-parameter subgroups are of the form 

Q(f) = exp(fA) (28) 

for some A G so(n) (which depends on the particular subgroup) and f G M. The following 
lemma will prove useful for the particular one-parameter subgroups we will consider. 

Lemma 1: Suppose that A G M(n) is such that A^ = —1„, and let Q(f) = exp(fA). Then 

Qit) = cos(f)I„ -I- sin(f)A. (29) 

Proof: A straightforward computation using the definition of the exponential map yields 

f^ Pi? 

exp(fA) = I„ -f tA H — -1 -h ■ ■ ■ (30) 

= + + (31) 

= cos(f)I„ -f sin(f)A (32) 

as claimed. ■ 

In the same way that e** = cos(f) -I- where = —1 traces out the unit circle, the 

family exp(fA) = cos(f)I„ -f sin(f)A where A G so(n), A^ = —1„ traces out a circle in the 
group SO{n). Thus if one can find a desirable skew-symmetric matrix A, optimizing over all 
of SO{n) can be reduced to optimizing over a circle, which can be done by brute force. 

While computing the exponential of an arbitrary matrix can be computationally demanding, 
the above proposition reduces us, in the case where A^ = —1„, to computing cos(f) and sin(f). 
This is useful when computing the optimal f G [0, 27r]. 

B. Gradient Fields and Steepest Descent on SO{n) 

Let / : M(n) ^ R be a differentiable real-valued function, and suppose that we prescribe 
coordinates q^, 1 < f, j < n to the space M{n). The gradient of / as a function on M{n) can 
be identified with the vector field 

V/ : M(n) ^ M(n), V/(Q) = , Q = (%)i<M<n (33) 
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We wish to understand eritieal points of / when restrieted to SO{n), that is, deseribe the gradient 
field V(/| 5 o(n))- To that end, let us also define the veetor field 

Xf : 50(n) ^ 50(n), X^(Q) = V/(Q)Q* - QV/(Q)* (34) 

whieh is essentially a translated version of V(/| 50 (n))> but is more eonvenient to work with sinee 
Xf(Q) is skew-symmetrie. The following theorem and its proof ean be found in [fTOl §2.4], in 
whieh the authors work with a more general strueture known as the Stiefel manifold. In the 
notation of ifTOll . the Stiefel manifold is equal to the orthogonal group when p = n. 

Theorem 1: ttlOl §2.4.4] Let / ; M{n) M be a smooth, real-valued funetion. Then for any 
Q G SO{n), we have 

V(/|5o(n))(Q) = XKQ)Q (35) 

and thus Q is a eritieal point of / if and only if 3fj(Q) = 0. 

Proof: See lfT0]| . ■ 

Let / : M(n) —)■ M be an objeetive funetion whieh we wish to minimize on the subset 
SO{n) C M{n). To do this, we use the conjugate gradient method of [[TOl §3.4], also known as 
the method of geodesic flow. The eonjugate gradient algorithm approximates the shortest path 
in SO{n) between an initial point Qo and an approximate loeal minimum Q^r of /. 

The eonjugate gradient method presented in lfT0]| works on a mueh more general elass of 
manifolds, but we simplify and summarize the method briefly here for the speeifie ease where 
the manifold is SO{n). The eonjugate gradient method starts with a rotation matrix Qo and a 
step size h, and reeursively updates via the formula 

Qfc+i = exp {-hXf{Qk)) Qk (36) 

The matrix Xf{Qk) is skew-symmetrie and should be thought of as an infinitesimal gradient 

matrix living on the Lie algebra so(n). Thus exp {—hXf{Qk)) is an element of SO{n), whieh 
restriets this gradient seareh algorithm to the spaee of rotation matriees as desired. For an 
explanation of why this algorithm finds a loeal minimum of /, see lfT0]| . Note that we do not 
assume / to be eonvex, and henee we must eontent ourselves with finding loeal rather than 
global minima of our objeetive funetions. 

Unlike many approaehes to optimization over rotation matriees, the above numerieal approaeh 
avoids the use of explieit parameterizations of rotation matriees. Sueh parameterizations beeome 
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unwieldy and non-canonical as the dimension n grows, henee we prefer the above method whieh 
eireumvents any attempt at parameterizing the speeial orthogonal group as is done in [|7]|- 

VI. Families of 2^-Dimensional Rotations 

Given a fixed eonstellation A” C M"' and some E^/Nq, finding a rotated version of A whieh 
optimizes the eutoff rate (fT^ is equivalent to finding maxima of the funetion / : —)■ M 

defined by 

/(Q) = i?(QA) (37) 

In this seetion we will eonstruet a family of eandidates for good rotations for arbitrary eonstel- 
lations A C for any k. We first use the numerieal eonjugate gradient method of (l3^ to 
find example eritieal points, and then demonstrate how to explieitly eonstruet sueh families of 
matriees in general, whieh approximate loeal maxima of the eutoff rate. 

If A C is an M-QAM eonstellation or any rotated, non-uniform variant thereof, we will 
refer to A” C as a 2nD M-QAM eonstellation. Thus by a 4D 4-QAM eonstellation, for 
example, we mean the set 

A = {(xi,..., X 4 ) G I Xj = ±1}. (38) 

of size 16. An M-QAM eonstellation in thus eontains M" distinet signal points. 

A. Finding Optimal Rotations Using Steepest Descent 

Exeept for some trivial eases (for example |A| = 2), analytieally determining the eritieal 
points of / from (1371) seems to be too diffieult a task. On the other hand, the eomputational 
simplieity of evaluating the eutoff rate (as opposed to working direetly with the CM eapaeity) 
allows one to use fairly eomplieated numerieal sehemes to find loeal maxima of / on SO{n). 
Thus to obtain good rotations of non-uniform and high-dimensional eonstellations, we ean use 
the eonjugate gradient algorithm of the previous subseetion. 

Consider the 4D 4-QAM eonstellation of (IMI) . For Eb/N^ between 8 and 12 dB, we ran 
the eonjugate gradient seheme (l3^ to numerieally ealeulate optimal rotation matriees for this 
eonstellation. For E^/Nq = 10 dB, for example, the geodesie flow algorithm outputs a rotation 
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matrix Q^v such that 



0 

0.73 

0.73 

0.73 

log(Qiv) = 

-0.73 

0 

0.72 

-0.72 

-0.73 

-0.72 

0 

0.72 


-0.73 

0.72 

-0.72 

0 


Here the initial point Qo was chosen to be the matrix Qq = exp(H), H = (h^), = 10“^ 

for i > j, hij = —hji for i < j, and ha = 0 for alH = 1,..., n. Experiments suggest similar 
small perturbations of the identity matrix work well as initial points for the conjugate gradient 
method. Similarly structured matrices were obtained for other values of E^/Nq and other X, 
namely, that log(QAr) appears to be a constant value times a matrix which depends only on the 
dimension n. 

These numerically-obtained results suggest that to construct such matrices explicitly instead 
of running the gradient descent scheme (l3^ for every value of Eh/No, one should construct a 
single skew-symmetric matrix A and consider exponentiating all of its scalar multiples. 


B. Explicit Construction of Rotation Matrices 


We now demonstrate how to construct families of matrices of the form log(Q 7 v) as in (l39l) . 
when the ambient space has dimension a power of 2. To this end we will construct a one- 
parameter subgroup of SO(2^), which in turn reduces the dimension of the search 

space from dim S'O(n) = — n)/2 down to one. This saves significant computation time 

for dimensions as small as n = 4. 

First let us recall the definition of the Hadamard matrices H 2 fc G M{2f)-. 


Hi = [1], = 




to 

1 

1 

to 

1 


(40) 


We now construct skew-symmetric matrices A 2 fc G so(2^) for A; > 1 recursively in the following 
way: 


Bl — [0], 52*: 


62*:—1 


5 ^ 2 *; —1 


H 2*:~1 


^ 62 *: — ! 


(41) 


A 2 . = (2^ - 1)-^/2B2* 


(42) 


The factor of (2^ — 1) is only a convenience that simplifies some expressions in what follows. 
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Given a fixed n = 2^, we consider the one-parameter family of rotation matrices 

Q„(t) = exp(tA„) e S'O(n) (43) 

for t G M. The following proposition describes the matrices Qn(f) explicitly, and relieves us of 
the computational burden of computing a matrix exponential. 

Proposition 1: For any dimension n = 2^ we have 

Quit) = cos(f)In + sin(f)A„. (44) 

Proof: One can show easily by induction that the matrices B„ and anti-commute, that 
is, that 

= -H„B„ (45) 


for all n > 2. Furthermore, it is well-known and easy to show that the Hadamard matrices satisfy 
K = nl„. It is now straightforward to show that B^ = — (n — 1)1^, or in other words that 

A^ = —In- (46) 

The result now follows from Lemma [B ■ 

The previous proposition also allows us to restrict the parameter space to f G [0,7r/2], as 
every matrix Qn(f) for t G (7r/2,27r] can be obtained from a matrix Qn{t) for t G [0,7r/2] via 
negation and taking transposes, operations which do not affect constellation performance. For 
any constellation A C M" and a fixed value of Eb/No, we can now optimize the cutoff rate with 
respect to rotation by computing 


topt = argmax i?(Q„(f)A) 

te[0,7r/2] 


(47) 


Q2(i) — 


(48) 


by simple exhaustive search. 

When n = 2 we have 

cos(f) sin(l:) 

-sin(f) cos(f) 

and hence the family Q 2 (t) is simply the whole group S'0(2). Finding the optimal rotation simply 
reduces to exhaustive search over the interval t G [0,7r/2], which was exactly the procedure 
carried out in Section ITTT-Bl 


When n = 4 one can see easily that Q4(0.72) is approximately the matrix Qjy from (l39l) . 
The transpose of the matrix Q4(0.56) was considered in conjunction with A a 4D 4-QAM 
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constellation in the DVB-NGH standard IfTSlI to minimize the bit error rate at the demapper. In 
a sense, one eould deseribe our method as a generalization of the rotation matriees eonsidered 
by DVB-NGH. 


C. Optimal Rotations at Low SNR 

Let us eonsider an arbitrary M-QAM eonstellation X C M" for n = 2^. Our goal in this 
subseetion is to find good rotations among the family Qn{t) when the SNR is low. At low SNR, 
errors are mostly eaused by large noise vectors as opposed to deep fades. One make this more 
preeise by ehoosing x ^ y E X and expanding the reeiproeal of the produet in the expression 
(fTOl) for the eutoff rate. We obtain 


2 = 1 


n 1+ 


(Xi 


Vi 


= 1 + 


y|li 


+ o 




(49) 


8Nq J 8Nq 

Thus for large A"o, the dominant term in the above is proportional to ||x —yl^, and the dominant 
terms in the sum (fTO are those for whieh the Euclidean distance is smallest. Hence if we transmit 
X and deeode x 7 ^ x, it is overwhelmingly likely that ||x — x ||2 = 2 , whieh is the minimal 
Euelidean distance between any two QAM constellation points. 

The previous paragraph suggests that it suffices to eonsider the approximation 


R{X) ^ R{X,2) 


(50) 


of the eutoff rate by the loeal eutoff rate. Moreover, this approximation is eonvenient beeause it 
allows us to compute the rotation parameter t optimizing R{Qn{t)X,2) explieitly. 

Proposition 2: Eet X C M" be a QAM constellation and let n = 2^. The funetion / : 
[0,7r/2] —)■ M defined by 

f{t) = R{Qr^{t)X,2) (51) 

has a single maximum at t = arccos(l/v^). 

Proof: Eet Q G SO{n) be any rotation matrix and consider the summation inside R{QX, r) 
defined by 

n 

E E IlT-iiEiE- ‘52) 

xEQA' yG_B(x,r) ^—1 8 A^o 

y/x 

Eor a fixed x G Tf, we have y E B{x, 2) if and only if x — y = ±2e for some standard basis 
vector e G M". Note that if e = where j = 1, ... ,n then Qx — Qy is the eolumn of Q, 
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whose coordinate is qij. Since Q is orthogonal and hence distance-preserving, it follows that 
Qi?(x, r) = -B(Qx, r) and hence 

1 


sw = E E n 


(53) 


xsa" y£B{x,r) *=1 1 “f 


2No 


By the symmetry of the constellation, one can see that for all j the product nr=i i+q^./ 2 No 
appears in the above summand the same number of times. Now setting Q = Qn(f) we obtain 
for some positive integer c the expression 

n n ^ 

1 cn 


=c5^n 


=1 1 ^ 


ait) 


(54) 


where g{t) is the function 

9{t) = 


1 + 


cos^(f) 

2iVn 


sin^\ 

{n-l)2NoJ 


n—1 


(55) 


We see that maximizing i?(Q„(f)A’, 2) with respect to t is equivalent to maximizing g{t). One 
can compute the derivative of g{t) explicitly to see that it has a single maximum on the interval 
t e [0,7r/2] at f = arccos(l/v^) which completes the proof. ■ 

We should expect from Proposition [2] that t = arccos(l/v^) is a decent approximation of the 
optimal t at low SNR, whenever R{X,2) is a decent approximation of R{X). This is confirmed 
via experiment for n = 2 in Fig. [2] and as we shall see in the next section, for n = 4 in Fig. H] 
and for n = 8 in Fig. |7l 


D. Diversity of the Constellations Qn{t)X 

Here we present a short proof of the fact that if X C M” is a QAM constellation, then 
the rotated constellation Qn{t)X is locally fully diverse for r = 2 and all t e (0,7r/2). 
Furthermore, we show by example that these rotated constellations lack full diversity. We will 
show in the next section that our locally fully diverse constellations can outperform the globally 
diverse constellations defined by number-theoretic rotations of [|3l|, lf2^ . even though our rotated 
constellations are not globally fully diverse. 

Proposition 3: Let A’ C M” be a QAM constellation and let Q = {qij)i<ij<n G SO{n) be 
any rotation matrix. Then QX satisfies L{QX, 2) = n if and only if qij 0 for all i, j. 

Proof: If X G A” and y G B{x^ 2) is not equal to x, then x — y = 2e for some standard 
basis vector e. Conversely, every standard basis vector of M"' can be expressed as e = 4(x — y) 
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for some x G A” and y G B{x,2). Hence QA being locally fully diverse for radius r = 2 is 
equivalent to no entry of Qe being zero for any standard basis vector e. But this is exactly the 
statement that qij ^ 0 for alH, j. ■ 

For any choice of t G (0,7r/2), it is now clear from Propositions [H and [3] that for A C M” a 
QAM constellation with n = 2^ the rotated version will satisfy 


L(Q„(f)A,2) = n 


(56) 


that is, the rotated constellation will be locally fully diverse. In fact, Proposition [3] essentially 
says that it is very easy to achieve local full diversity; simply pick a rotation matrix without any 
zero entries, and such matrices are dense in SO{n). Thus the property of local full diversity is 
not particular to the family Qn(Q; rather, among all rotations which offer local full diversity, 
we study these because they were so frequently the outcome of the numerical algorithm (l3^ . 

However, the rotations Qn(f) do not offer full diversity when n > 2. Let us take, for example, 
n = 4, and consider z = x — y where x 7 ^ y G A. Then 


Zi 


azi + b[z 2 + ^3 + Z 4 ) 

Z 2 


az 2 + b{-zi + Z 3 - Z 4 ) 

^3 


az 3 + b(—Zi — Z 2 Z 4 ) 

_Z4_ 


az 4 b(—zi Z 2 — Z 3 ) 


(57) 


where a = cos(f) and b = sin(f)/\/3. Now notice that for zi = Z 2 = 0 and Z 3 = —Z 4 we 
have (Qn(f)z)i = 0. It is easy to check that the points x = (1,1,1, —1)* and y = (1,1, —1,1)* 
belonging to the standard 4D 4-QAM constellation will satisfy (Qn(f)z)i = 0, resulting in a 
rotated constellation that is not fully diverse. 


VH. Simulation Results for n = 4 and n = 8 

In this section we confirm the benefits of the family Qn(t) of rotation matrices for n = 4, 8. 
Our basis for comparison is always the optimal algebraic rotation of the same dimension, as found 
in [|26]| . With regard to an algebraic rotation, “optimal” means with respect to the normalized 
minimum product distance of the rotated constellation. Maximum-likelihood decoding was used 
for all error rate simulations. 
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A. Rotated Constellations in 


Let us further examine the family (1431) for the simplest non-trivial example, that of rotated 
constellations in It follows from Proposition [U that 


Qi(t) 


, sin(f) 

COs(t)l4 H- -j=- 

v3 


0 111 

-1 0 1-1 

- 1-1 0 1 

-1 1-1 0 


(58) 


We note that this one-parameter subgroup of SO{n) was implicitly considered by the DVB-NGH 
standard [fTSl §6.4] when rotating 4D 4-QAM constellations. The DVB consortium specifies the 
particular rotation parameter t by instead specifying the parameter sin(f), but this difference is 
immaterial. 

For 4D M-QAM and M-NUQAM and M = 4,16, 64, we computed the optimal rotation 
matrix Q 4 (fopt) by exhaustive search over the interval t G [0,7r/2] as in (1471) and plotted fopt 
in degrees as a function of Eh/No in Fig. |4| When M = 64, note the range of relatively 
low E},/Nq values for which fopt = arccos(l/\/4) = 60°, as predicted by Proposition [2l The 
NUQAM constellations in this simulation were first optimized on a per-SNR basis, and then 
optimal rotations were computed. 




Fig. 4. On the left, the optimal rotation parameter fopt as in (|47j as a function of Eh/No, for 4D M-QAM and M-NUQAM, 
M = 4,16, 64. On the right, the improvement in R{X) for 4D M-QAM and M-NUQAM constellations for M = 4, 16, and 
64 when using the rotation matrices Q 4 (fopt) as opposed to the Krtiskemper rotation, which is the optimal algebraic rotation in 
R'^ (26). 
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To compare our method with the optimal number-theoretie eonstruetion, we also plot in Fig. 
m the improvement 

Ai?(T’) = i?(Q4(topt)^) - R{KaX) (59) 

of our method over the algebraie method. Here K 4 is the Kriiskemper rotation, whieh is the 
optimal algebraie rotation of and whose generator matrix is available at ll26ll . Our rotations 
Q 4 (fopt) offer modest but eonsistent improvement in eutoff rate over the Kriiskemper rotation at 
almost every value of Eb/No and every Tf C we investigated. Despite the small improvement 
in eutoff rate at high SNR, this is the regime where the rotations Q 4 (fopt) offer the highest error 
performanee gains over K 4 , as we see below. 



Fig. 5. Bit error rates for the family Q 4 (fopt) for M-QAM and M-NUQAM in R'^, for M — 16, 64. The Kriiskemper rotation, 
which is the optimal algebraic rotation in R"*, served as the basis for comparison and is plotted with the analogous dashed black 
curves. Simulations were performed by modulating approximately 10® bits at each value of Eb/No. 


We plot in Fig. Obit error rates of the family Q 4 (fopt) for M-QAM and M-NUQAM, for 
M = 16, 64. Bit strings were assigned aeeording to the Gray labeling of the underlying QAM 
symbols. The rotations Q 4 (fopt) outperform the Kriiskemper rotation for both uniform and non- 
uniform eonstellations at all values of Eb/No. When M = 16, we see an inerease in performanee 
of up to 2 dB for non-uniform eonstellations and about 1.5 dB for uniform eonstellations. When 
M = 64, the inerease is approximately 1 dB for uniform eonstellations, with a smaller inerease 
in performanee for non-uniform eonstellations. 

While the operating SNR is very high, we have provided plots in this regime to study 
asymptotie behavior (i.e. diversity). This allows for fair eomparison with the algebraie rotations 
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of [fT^ which are specifically tailored for the asymptotic regime. The crucial point is not the 
magnitude of the gains over the fully-diverse algebraic rotations, but rather that there are any 
gains at all in the absence of full diversity even in the asymptotic regime. Thus Fig. [ 5 ] underscores 
the original theoretical observation motivating local diversity, namely that full diversity is too 
restrictive a notion even in the very high SNR regime. In fact, one can show easily that 
L{Q,i{topt)X, cx)) = 3 for all fopt in the given Eb/No range, that is that the rotations outperform 
the Kriiskemper rotation even though their global diversity is in some sense deficient. 




Fig. 6 . Local (with r = 2) and global minimum product distances dp(X,r) for the constellations rotated algebraically with 
the matrix K 4 , and with the family Q 4 (fopt). In the left figure are the results for 4D 16-QAM, and in the right figure are the 
results for 4D 64-QAM. 


Finally in Fig. 0 we plot the various local and global minimum product distances dp{QX, 
for r = 2, CX) as defined in Section IIV-B[ for both Q = K4 and Q = Q4(fopt)7 and X = ^ 
M-QAM for M = 4,16. We see that for M = 16, both minimum product distances correctly 
predict that Q 4 (fopt) outperforms K 4 . However when M = 64, the local minimum product 
distance correctly predicts that the family Q4(fopt) outperforms the rotation K4 for almost all 
values of Eb/No, whereas the global minimum product distance effectively makes a mistake for 
almost all values of Eb/No. Hence the local minimum product distance seems a more appropriate 
measure of the performance of these constellations. 
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B. Rotated Constellations in 

In this subsection we study rotations of the 8D 4-QAM constellation A’ C M®. In Fig. |7] 
we plot the value of fopt as a function of Eb/No. As predicted by Proposition [21 we obtain 
fopt = arccos(l/\/8) = 69.2952° for several low values of Eb/N^, approximately in the range 
4-7 dB. We also plot in Fig. |7]the improvement 

= i?(Q8(V)A') - R{C^X) (60) 

where Cg is the optimal algebraic rotation in [|26l. We see from the plot that in the range 
of Eb/No = 4-14 dB, the family Q8(fopt) shows modest but consistent improvement over Cg in 
terms of cutoff rate, while the algebraic rotation has better cutoff rate outside of this region. 




Fig. 7. On the left, optimal rotation parameter topi as in l l47t as a function of Eb/No, for 8D 4-QAM. On the right, improvement 
in R{X) for 8D 4-QAM when using Q8(topt) as opposed to the algebraic rotation Cs of 1261 . which is the best-known algebraic 
rotation of E®. 


In Fig. I^we plot the bit error rate of the rotations Q8(fopt) as well as those of the optimal 
algebraic rotations of M” for n = 5,6,7, 8, denoted by C 5 ,M 6 ,K 7 , Cg, respectively In 
the low SNR regime the performance of the two schemes is identical. Beyond Eb/N^ > 12 dB 
(corresponding to a bit error rate of approximately 10“^) the algebraic rotation outperforms the 
rotations Qg(fopt)- 

While the rotations Qg(fopt) have inferior error rate performance compared to Cg at high SNR, 
it can be shown that the global diversity of our scheme is only L(Q8(fopt)'^, cxo) = 5 whereas 
the fully-diverse algebraic rotations of achieve global diversity n. Thus the error plots show 
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Fig. 8 . Bit error rates for the family Q 8 (fopt) for 4-QAM in R®. The optimal algebraic rotations of R" for n = 5, 6 , 7, 8 , 
denoted respectively by C 5 , Me, K 7 , Cs, served as the basis for comparison 1261 . Simulations were performed by modulating 
approximately 10® bits at each value of Eh/No according to the Gray labeling on the underlying per-coordinate constellation. 
A 2-PAM (pulse amplitude modulation) constellation was used per real axis for the odd-dimensional constellations, to ensure 
fair comparison across dimensions. 


that Qsitopt) outperforms rotations whose global diversity is up to n = 7, lending credenee to 
the original theoretieal notion that local diversity should be a superior predictor of performance. 

Lastly, we mention that neither local nor global minimum product distances appeared to predict 
the relative error performance, hence we omit plots of these quantities. 

VIII. Conclusions and Future Work 

We have constructed a family of rotation matrices Q 2 k{t) G SO (2^) for every k with the goal 
of maximizing the cutoff rate of arbitrary constellations in M". Our approach is an adaptive per- 
SNR optimization, in which an optimal rotation for a given constellation and SNR value can be 
computed by a simple exhaustive search on the interval [0,7r/2]. Our rotations are applicable to 
non-uniform and uniform constellations alike, thereby improving the ability of non-uniformity to 
improve the cutoff rate. The cutoff rate has essentially served as a proxy for, and lower bounds, 
the CM capacity of the constellation. The computational intractability of the CM capacity for 
the channel under consideration makes the cutoff rate a much more attractive objective function. 

Our rotations produce constellations which, while lacking full diversity, are “locally” fully 
diverse in the sense that every point is distinguishable from all of its closest neighbors by a 
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single coordinate. We have thus defined a new notion of diversity, deemed local diversity, which 
apparently suffices to guarantee modest improvement in both cutoff rate and error performance 
over the number-theoretic methods of ^ when the dimension is n = 4. By constructing a well¬ 
performing one-parameter subgroup of rotation matrices, we have added a degree of flexibility 
to the use of rotated constellations. We have used this parameter to optimize with respect to 
SNR, but one may use it to optimize with respect to, for example, outer code rate as in the 
DVB-NGH standard ifBl . 

Future work consists of generalizing our construction to create well-performing families of 
rotation matrices Qn{t) for all n, not just those which are a power of 2 , and additionally 
improving the performance of our scheme for dimension n = 8 . The optimal parameter t 
should also be investigated in connection with outer error-correcting codes. Secondly, we wish 
to study the decoding complexity of our rotation matrices, and perhaps construct some variants 
which offer provably simpler sphere decoding complexity. Lastly, we plan on investigating 
local diversity from an information-theoretic perspective to explain exactly why locally fully 
diverse constellations can outperform those which are globally fully diverse. We hope that the 
resulting intuition can be applied to the study of other channels, for example MIMO channels and 
Rician channels, and not simply the SISO Rayleigh fading channel. We believe the robustness 
of our approach, especially the general independence of the numerical methods on the particular 
objective function we used, will prove that our ideas will be similarly useful in other settings. 
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